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ABSTRACT 

A scheme that publishes aggregate information about sen- 
sitive data must resolve the trade-off between utility to in- 
formation consumers and privacy of the database partici- 
pants. Differential privacy [5] is a well-established definition 
of privacy — this is a universal guarantee against all attack- 
ers, whatever their side-information or intent. In this paper, 
we present a universal treatment of utility based on the stan- 
dard minimax rule from decision theory [13] (in contrast to 
the utility model in [8], which is Bayesian). 

In our model, information consumers are minimax (risk- 
averse) agents, each possessing some side-information about 
the query, and each endowed with a loss-function which 
models their tolerance to inaccuracies. Further, informa- 
tion consumers are rational in the sense that they actively 
combine information from the mechanism with their side- 
information in a way that minimizes their loss. Under this 
assumption of rational behavior, we show that for every fixed 
count query, a certain geometric mechanism is universally 
optimal for all minimax information consumers. Addition- 
ally, our solution makes it possible to release query results 
at multiple levels of privacy in a collusion-resistant manner. 

1. INTRODUCTION 

Privacy Mechanisms: Agencies such as medical establish- 
ments, survey agencies, governments use and publish aggre- 
gate statistics about individuals; this can have privacy im- 
plications. Consider the query: Q: How many adults from 
San Diego contracted the flu this October? The government 
can use the query result to track the spread of flu, and drug 
companies can use it to plan production of vaccines. How- 
ever, knowledge that a specific person contracted the flu 
could be used to deny her health insurance based on the 
rationale that she is susceptible to disease. As discussed 
in [20], and as is exemplified by [21, 16], seemingly benign 
data publications can have privacy implications. Thus, it 
is important to think rigorously about privacy. The frame- 
work of differential privacy [3] does this, and is applicable 



widely (see Section 2.8). 

Mechanisms guarantee differential privacy by perturbing re- 
sults - they add random noise to the query result, and guar- 
antee protection against all attackers, whatever their side- 
information or intent (see Section 2.1 for a formal definition). 

Our Utility Model: The addition of noise increases pri- 
vacy but intuitively reduces utility of the query result. To 
understand this privacy-utility trade-off, we propose a for- 
mal decision-theoretic model of utility. Decision-theory is a 
widely applied field that provides mathematical foundations 
for dealing with preferences under uncertainty. The use of 
decision theory in this context is appropriate because, as we 
discussed above, mechanisms guarantee differential privacy 
by introducing uncertainty. 

In our model of utility (see Section 2.3 for details), the 
user of information, i.e. the information consumer has side- 
information — for instance, knowledge of the population of 
San Diego is an upper bound on the result of the query Q. 
It has a loss-function that expresses it's tolerance to inac- 
curacy. It is rational in the sense that it combines informa- 
tion from the mechanism with its side-information optimally 
with respect to its personal loss-function. It is risk-averse 
in the sense that it would like to minimize worst-case loss 
over all scenarios. 1 

Given the privacy parameter, the loss-function and the side- 
information of an information consumer it is possible to 
identify an optimal mechanism - a mechanism that is differ- 
entially private and that maximizes its utility. See Section 
2.4.3 for an algorithm to find such a mechanism. 

Non-Interactive Settings: Very often aggregate statis- 
tics, like answers to Q, are published in mass media as op- 
posed to following a query-response form [18]. In such cases 
neither the information consumer nor it's loss-function and 
side-information are known in advance. Thus it seems hard 
to identify the optimal mechanism for a information con- 



Nevertheless, we show that it is possible to deploy an opti- 
mal mechanism without knowledge of the information con- 
sumer's parameters. Furthermore, this mechanism is uni- 



1 Ghosh et al. [8] propose a model with most of these fea- 
tures, but assumes that information consumers are Bayesian 
and have a prior over the query-result. 
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versally optimal for all information consumers, no matter 
what their side-information or loss-function. 

How can we identify the optimal mechanism without knowl- 
edge of the information consumer's parameters? The ap- 
parent paradox is resolved by relying on the information 
consumers' rationality, i.e., each information consumer uses 
its personal loss-function and side-information to actively 
transform the output of the deployed mechanism. For a 
certain class of queries called count queries, when the de- 
ployed mechanism is a certain geometric mechanism, this 
transformation is effective enough to result in the optimal 
mechanism for the information consumer — a fact that we 
will establish via linear-algebraic proof techniques. 

Multiple Levels of Privacy: We also show how to si- 
multaneously release the query result at different levels of 
privacy to different information consumers. This is useful, 
for instance, when we want to construct two versions of the 
report on flu statistics, one which prioritizes utility for the 
eyes of government executives, and a publicly available In- 
ternet version that prioritizes privacy. 

A naive solution is to perturb the query results differently, 
independently adding differing amounts of noise each time. 
The drawback is that consumers at different levels of privacy 
can collude and combine their results to cancel the noise (as 
in Chernoff bounds). An alternate way is to correlate the 
noise added to different outputs. We give an algorithm to 
achieve this that makes the data release collusion-resistant. 

In this paper we focus on a single query; the complexity 
comes from a rich model of consumer preferences, where we 
consider different utility functions for each consumer and 
optimize for each of them. [1, 10, 9] exploit similarities 
between the queries to obtain extension to multiple queries 
with good utility guarantees. However, they do not consider 
a rich consumer preference model. Our results could be used 
as a building block while answering multiple queries. 

2. MODEL AND RESULTS 

We gave a informal description of our model and results 
in the Introduction. In this section, we formally define our 
model and discuss the main results. The proofs of the results 
are presented in Sections 3, 4. 

2.1 Privacy Mechanisms and Differential Pri- 
vacy 

A database is a collection of rows, one per individual. Each 
row is drawn from an arbitrary domain D; for instance, in 
our running example, a row of the database has the name, 
age, address and medical records of a single individual. A 
database with n rows is thus drawn from the domain D n . 

We will focus on a class of queries, called count queries, 
that frequently occur in surveys: Given a predicate p : D — > 
{True, False}, the result of a count query is the number of 
rows that satisfy this predicate, a number between and the 
database size, n. Q is an example of a count query with the 
predicate: individual is an adult residing in in San Diego, 
who contracted flu this October. Though simple in form, 
count queries are expressive because varying the predicate 



naturally yields a rich space of queries. 

We guarantee differential privacy to protect information of 
individual database participants. Differential privacy is a 
standard, well-accepted definition of privacy [3] that has 
been applied to query privacy [7, 5, 17], privacy preserv- 
ing machine learning [1, 11] and economic mechanism de- 
sign [15] . A fixed count query maps the database d to a num- 
ber which belongs to the set N. A privacy mechanism M 
for a fixed count query is a probabilistic function that maps 
a database d € D n to the elements of the set N = {0 . . . n}. 
These can be represented, for each d € D n , by {md, r }reN, 
which gives for each database d £ D n the probability that M 
outputs r. For the database d, the mechanism releases a per- 
turbed result by sampling from the distribution {m djr } r6 jv. 

The Geometric Mechanism [8] is a simple example of a pri- 
vacy mechanism. It is a discrete version of the Laplace 
Mechanism from [5]. 

Definition 1 (a-Geometric Mechanism). When the true 
query result is f(d), the mechanism outputs f(d) + Z. Z 
is a random variable distributed as a two-sided geometric 
distribution: Pr[Z = z] = y^a' z ' for every integer z. 
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Figure 1: The probability distribution on outputs 
given by the Geometric Mechanism for a = 0.2 and 
query result 5. 

Informally, a mechanism satisfies differential privacy if it in- 
duces similar output distributions for every two databases 
that differ only in a single individual's data, thereby ensur- 
ing that the output is not sensitive to any one individual's 
data 2 . Formally, differential privacy is defined as follows [5]: 

Given a privacy parameter a € [0, 1] and two database 
d\,d2 £ D n that differ in at most one individual's data, 
a mechanism M is a- differentially private, if for all elements 
r in the range of the mechanism : --Xd 1; r > %d 2 ,r > a-Xd 1; r- 



2 Thus any attack on an individual's privacy that can be 
constructed using the perturbed query result with this indi- 
vidual present in the database can also be constructed, with 
a similar success rate, without this individual present in the 
database. See See [5, 12] for details of such semantics of 
differential privacy. 
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The parameter a can be varied in the interval [0, 1] to vary 
the strength of the privacy guarantee — when a = 0, the 
above definition is vacuous and there is no privacy, whereas 
when a = 1, we effectively insist on absolute privacy- the 
query result cannot depend on the database because we re- 
quire distributions over perturbed results to be identical for 
neighboring databases. 

2.2 Oblivious Mechanisms 

We will focus in this paper on a class of privacy mechanisms 
that are oblivious. A mechanism is oblivious if it sets up an 
identical distribution over outputs for every two databases 
that have the same unperturbed query result. Naturally, 
an implementation of an oblivious mechanism need only 
have access to the true query result — the input — and can 
be oblivious to the database itself. An oblivious mechanism 
for count queries can be expressed by the set of probabil- 
ity masses for every i G N : {a^rjrgiv, where Xi :r is the 
probability that the mechanism outputs r when the true 
result is i. Appendix A shows that this restriction to oblivi- 
ous mechanisms is without loss of generality. The geometric 
mechanism (Definition 1) only depends on the query result 
f(d) and not on the database d itself; so it is a oblivious 
mechanism. 

The query result for a count query can change by at most 
one when we change any one row of the database, so we can 
rewrite the definition of differential privacy as follows: 

Definition 2 (Differential Privacy for Count Queries). An 

oblivious mechanism for count queries for a € [0, 1] is a- 
differentially private if for all i G {0 . . . n — 1}, r G N : 

a 

Observe that the geometric mechanism is a-differentially 
private because for two adjacent inputs i, i + 1 G N, and 
any output r G N, G [a, 1/a]. 

2.3 Minimax Information Consumers 

We now discuss our model of an information consumer's util- 
ity. The loss-function l(i,r) : N x N R specifies the loss 
of the information consumer, given the mechanism outputs 
r when the true result is i. We only assume that the loss- 
function is monotone non-decreasing in \i — r\, for every i. 
That is, the consumer becomes unhappier as the returned 
answer is further away from the true result. 

Consider some examples of valid loss-functions: The loss- 
function l(i,r) — \i — r\ quantifies the mean error — for our 
query Q, this loss-function may be a reasonable one for 
the government who want to keep track of the rise of flu. 
The loss-function l(i,r) — (i — r) 2 quantifies the variance 
in the error — this may be reasonable for a drug company 
who wants to ensure that they don't over-produce or under- 

{0 if i = r 
1 if i r 

measures the frequency of error. 

Additionally, we will assume that the information consumer 
has side information S C N, i.e., the information consumer 
knows that the query result cannot fall outside the set S. For 



instance, knowledge of the population of San Diego yields an 
upper-bound on the query result. The drug company may 
also know how many people bought its flu drug this month, 
yielding a lower bound on the query result. 

For any specific input i, the loss- function I allows us to eval- 
uate the information consumer's dis-utility as the expected 
loss over the coin tosses of the mechanism: ~^2 rGN l(i, r)-Xi tT . 
To quantify the overall loss, we follow the minimax decision 
rule, i.e., we take the worst-case loss over all inputs in the 
set S [13]. This amounts to the information consumers be- 
ing risk-averse. Hence, the dis-utility of the mechanism x to 
the consumer c is : 

L(x) = max l(i, r) ■ x i:r (1) 

rgJV 

2.4 Interactions of Information Consumers with 
Mechanisms 

As mentioned in the Introduction, information consumers 
actively interact with the mechanism to induce a new mech- 
anism; we now discuss the mechanics of this interaction. 

2.4.1 Motivation 

The following example argues why a rational information 
consumer will not accept the mechanism's output at face 
value. 

Example 1. Recall the query Q defined in the Introduction. 
Suppose that the information consumer is a drug company, 
who knows that I individuals in San Diego bought its flu drug 
in the month of October. Thus the query result Q must be 
at least I; the information consumer cannot conclude that 
the query result is exactly I because some individuals with 
flu may have bought a competitors drug, or bought no drug 
at all. Thus it has side-information S = {I . . . n} . 

Suppose we deploy the geometric mechanism for the query Q. 
This mechanism returns with non-zero probability outputs 
outside the set {I . . . n}. Such outputs are evidently incorrect 
to the information consumer, and naturally it makes sense 
for the information consumer to map these results within 
the set {I . . . n}. Though it is not clear what the best way of 
doing so is, a reasonable rule may be to re-interpret results 
less than I as I, and results larger than n as n. 

2.4.2 Feasible Interactions 

Before we discuss the optimal way for an information con- 
sumer to interact with the mechanism, we describe the space 
of feasible interactions. On receiving a query result r from 
the mechanism, the consumer can reinterpret it as a different 
output. This reinterpretation can be probabilistic and can 
be represented by a set of probability masses {T r y : r' G N} 
which gives for each result r, the probability that the con- 
sumer will reinterpret it as the output r' . Such an inter- 
action induces a new mechanism for the user. Suppose 
the deployed mechanism is represented by the set of prob- 
ability masses {yt,r ■ i,r € N}, and the induced mech- 
anism as the probability masses {xiy : i,r € N}, then 
x i,r' — Sreiv V i < r ' T r , r ' ■ We formalize this in a definition. 

Definition 3 (Derivability). Given two mechanism x and 
y, we say that mechanism x can be derived from y if and 
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only if, for every r G N , there exists a set of probability 
masses {T r y : r' € N} such that for every i,r' € N : 

2.4.3 Optimal Interactions 

Given a deployed mechanism y, the optimal interaction T* 
is one that minimizes the information consumer's maximum 
loss on the induced mechanism. The optimal interaction can 
be computed using a simple linear program. There are n 2 
variables: one for each T* r , eN . The objective function is 
obtained by minimizing the loss to the consumer if it uses 
interaction T* . The constraints are obtained from the fact 
for each r, the entries T* r , form a probability distribution 
and hence sum up to 1 and that all entries of T* are positive. 
The actual linear program is given as : 



minimize 



reN 



max >^ Xi r ■ III, r) 
ies t—i 

reN 

Vi G 7V,Vr G N 
\/r€N 

Vr G N,\/r' G N 



2.5 



Optimal Mechanism for a Single Known 
Information Consumer 

Identifying the optimal mechanism for a specific consumer 
reduces to the following: Identify a level of privacy a with 
which to release the result. Find the consumer's loss-function 
and side-information. Identify an a-differentially private 
mechanism such that the mechanism induced by the con- 
sumer's optimal interaction (as described in the previous 
section), has the best possible utility. 

In the case of a single information consumer, we can obvi- 
ate the need for the information consumer to reinterpret the 
deployed mechanism's output: Suppose there is a mecha- 
nism y with post-processing T that induces a mechanism x. 
Clearly, presenting x directly to the information consumer 
yields at least as much utility for it. All we have to en- 
sure is that x is a-differentially private, and a simple proof 
(omitted) shows that this is indeed so. 

Thus, to identify the optimal mechanism for a specific infor- 
mation user, it suffices to search over a-differential mecha- 
nisms. For a given consumer c with loss-function 



L(l, S) = max x i>r ■ l(i, r) 



and privacy parameter a, the optimal differentially private 
mechanism M c is the solution to a simple linear program. 
Like in the previous section, there are n 
each matrix entry of the mechanism x. 
to minimize the user's loss function. The constraints are 
obtained by the facts that 



2 variables one for 
The objective is 



1. a; is differentially private. So the variables Xi 
satisfy Definition 2 



must 



2. For each input i, elements x itV form a probability dis- 
tribution and hence sum up to 1. 



3. All elements x iiT are positive 
Writing this as an optimization problem we get: 



minimize 



X% r Ct ' Xj^l )■ 

GC ' X% f Xj 1 -^-\ r 

^ ^ Xi,r — 1 



max > Xi r ■ l(i, r) 
ies ^ ' v ' 

reN 

Vi G N \ {n},Vr G N 
Vi G N\ {n},Vr G N 

\/ieN 



rgiV 



x iiT > Vi G N, Vr G N 

We can convert it into a Linear Program, the solution of 
which gives usi*. 



minimize 

d — x i,r ■ Kh r) > 

reN 

Xj, f Oi ' 3j%-\-\- , r ^ 

ct ■ Xi >r — Xi+\ iT < 

^ ^ Xi^r — 1 
reN 

Xi r > 



d 

Vi G S 

Vi G N\{n},Vr G N 
Vi G N \{n},Vr G N 

\/ieN 

Vi G N,W G N 



To deploy this mechanism x* , we first compute the true 
query result, say i, then sample the perturbed result r from 
the distribution {x* r : Vr G iV}, and release the sampled 
result. Table 1(a) gives an example of a optimal mechanism 
for a particular information consumer. 

2.6 Optimal Mechanism for Multiple Unknown 
Information Consumers 

How can we extend the results of the previous section to 
multiple consumers? The naive solution is to identify and 
separately deploy the optimal mechanism for each informa- 
tion consumer as described in the previous section. 

There are two reasons why this is undesirable. First, the 
naive solution results in the release of several re-randomizations 
of the query result — this allows colluding consumers to com- 
bine their results and cancel out the noise leading to a degra- 
dation in privacy; see [15] for a discussion, 

Second, solving the linear program that identifies the op- 
timal mechanism for a user requires the knowledge of the 
consumer's parameters; knowledge that is often unavailable 
when the decision of which mechanism to deploy is made. 
Consider a report published on the Internet. It is not clear 
who the information consumers are going to be. 

Our main result works around these issues successfully. 

Theorem 1. Consider a database d, count query q, k con- 
sumers and privacy levels ct\ < ... < a k . There exists a 
mechanism M that constructs k results ri . . . rt, and releases 
result ri to the ith information consumer, such that: 
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'2/3 5/17 1/25 1/98 
1/6 7/11 7/44 2/49 
2/49 7/44 7/11 1/6 
1/98 1/25 5/17 2/3^ 
(a)The Optimal Mechanism 



4/3 1/4 1/16 1/48 

1/3 1 1/4 1/12 

1/12 1/4 1 1/3 

1/48 1/16 1/4 4/3 

( b ) G 3,J 

Mechanism with access to the database. 



9/11 
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2/11 
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(c) Consumer Interaction 
Mechanism with access to the user parameters. 



Table 1: This shows the optimal mechanism for a consumer c with loss-function l(i,r) 
information S = {0, 1, 2, 3}. n = 3, a = 1/4. 



r\ and side- 



1. ( Collusion- Resistance) Mechanism M is a t i -differentially 
private for any set I of colluding information con- 
sumers who combine their results. Here, C C {1 ... k} 
and i' = min{j : j G C } . 

2. (Simultaneous Utility Maximization) Suppose that the 
ith consumer is rational and interacts optimally with 
the mechanism (as described in Section 2.4-3), then 
its utility is equal to that of the differentially private 
mechanism tailored specifically for it (the mechanism 
from Section 2.5). 

We now describe the release mechanism M. The ith stage of 
the mechanism Mi is just the a;-geometric mechanism. We 
shall prove in Lemma 3, that for any a > (J, the a-geometric 
mechanism can be derived from the /3-geometric mechanism: 
that is there is an implementable mechanism T a ^ such that 
if we use T a>i e to reinterpret results given by the /3-geometric 
mechanism, we get the a-geometric mechanism. The query 
results n are not generated independently of each other, 
they are obtained by successive perturbations: the result 
n of mechanism Mi is given as input to the mechanism 
Ti — T a(]Cti+1 . Hence, the (i + l)th stage mechanism M i+ i 
is just the a;+i-geometric mechanism. This specifies how the 
noise added to the query results is corelated. We describe 
the mechanism formally in Algorithm 1. In Section 4.1 we 
show that it is collusion-resistant. 

Consumer i interacts optimally with the published query 
result r; to get a result tailored specifically for it. In Sec- 
tion 4.2, we prove that the interaction yields optimal utility 
for the consumer. The main idea is that the optimal mecha- 
nism can be factored into two parts - The first is a database 
specific mechanism which has access to the database but 
not to the user parameters. In our case this is the a-- 
geometric mechanism. The second is the user specific mech- 
anism, which has access to the user loss-function and side- 
information and the perturbed query result (given by the 
first mechanism), but not to the database itself. Table 1 
shows these two factors of the optimal mechanism discussed 
in Section 2.5. 

We briefly discuss proof techniques: Section 3 completely 
characterizes mechanisms derivable from the geometric mech- 
anism using linear algebraic techniques. Section 4 applies 
this characterization twice: the first application shows that 
a a-geometric mechanism can be derived by re-randomizing 
the output of a /3-geometric mechanism so long as a > /3. 
The second application shows that the mechanism induced 
by the interaction of a rational information consumer with 
the geometric mechanism is an optimal solution to the linear 
program mentioned in Section 2.5. 



2.7 Comparison with Bayesian Information Con 
sumers 

An alternative to the Minimax decision rule is the Bayesian 
decision rule. Ghosh et al. [8] prove an analogous result 
to Theorem 1 for all Bayesian information consumers. We 
briefly compare the models and the proof techniques. 

The main distinction between the two models is their treat- 
ment of side-information. The Bayesian model requires agents 
to have a prior over all possible scenarios (true query re- 
sults). Often, in practice, agents do not behave consistent 
with the preferences of the Bayesian model, perhaps because 
they find it hard to come up with meaningful priors [14, Ex- 
ample 6.B.2,6.B.3], or are genuinely risk-averse [14, Section 
6.3]. 

As discussed in [8] , Bayesian information consumers employ 
deterministic post-processing, unlike minimax information 
consumers which employ randomized post-processing (For 
example, see Table 1). Handling this extra complexity re- 
quires us to construct a broader characterization of mech- 
anisms derivable from the geometric mechanism — Section 3 
presents a complete characterization in terms of a simple 
condition on the probability masses Xi-ij, Xij, Xi+ij. Our 
proof avoids the LP based techniques and counting argu- 
ments of [8], and consequentially strictly generalizes and 
gives a simpler proof of the main result of that paper. In 
addition, our characterization enables us to release data at 
multiple levels of privacy in a collusion-resistant manner. 

2.8 Related Work 

A recent thorough survey of the state of the field of differen- 
tial privacy is given in [4]. Dinur and Nissim [2], Dwork et 
al. [6] establish upper-bounds on the number of queries that 
can be answered with reasonable accuracy. Most of the dif- 
ferential privacy literature circumvents these impossibility 
results by focusing on interactive models where a mecha- 
nism supplies answers to only a sub-linear (in n) number of 
queries. Count queries (e.g. [2, 7]) and more general queries 
(e.g. [5, 17]) have been studied from this perspective. 

Hardt and Talwar [9] give tight upper and lower bounds on 
the amount of noise needed to ensure differential privacy 
for d non-adaptive linear queries, where the database is a 
vector in R n . Hay et al. [10] give a way to increase accuracy 
of answering multiple related queries while ensuring that the 
query results follow consistency constraints. 

Blum et al. [1] focus attention to count queries that lie in 
a restricted class; they obtain non-interactive mechanisms 
that provide simultaneous good accuracy (in terms of worst- 
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case error) for all count queries from a class with polynomial 
VC dimension. Kasiviswanathan et al. [11] give further re- 
sults for privately learning hypotheses from a given class. 

The use of abstract "utility functions" in McSherry and Tal- 
war [15] has a similar flavor to our use of loss- functions, 
though the motivations and goals of their work and ours 
are unrelated. Motivated by pricing problems, McSherry 
and Talwar [15] design differentially private mechanisms for 
queries that can have very different values on neighboring 
databases (unlike count queries); they do not consider users 
with side information and do not formulate a notion of mech- 
anism optimality (simultaneous or otherwise). 

Our formulation of the multiple privacy levels is similar to 
Xiao et al. [22]. However, they use random output per- 
turbations for preserving privacy, and do not give formal 
guarantees about differential privacy. 

3. CHARACTERIZING MECHANISMS 
DERIVABLE FROM THE GEOMETRIC 
MECHANISM 

In this section we give a characterization of all mechanisms 
that can be derived from the geometric mechanism. Recall 
that differential privacy imposes conditions on every two 
consecutive entries (xi,X2) of every column: x\ > ax2 (and 
X2 > ax\). Our characterization imposes syntactically simi- 
lar conditions on every three consecutive entries (xi, X2, X3) 
in a column: (X2 — a ■ X3) > a(xi — a ■ X2). Neither condition 
implies the other. This characterization is both necessary 
and sufficient for any differentially private mechanism to be 
derivable from the geometric mechanism. 

We defined feasible consumer interactions in Section 2.4.2. 
A slightly different way of representing these is to arrange 
the probability masses in a n x n matrix {T r y) r y eN . We 
say that a matrix is (row) stochastic if the sum of elements 
in each row is 1 and all elements are non-negative. We say 
that a matrix is a generalized (row) stochastic matrix if the 
if the sum of elements in each row is 1, but with no condition 
on individual entries. If the deployed mechanism is given by 
the matrix y, and the reinterpretation by the matrix T, then 
the new mechanism is given by the matrix x = y ■ T. 

We define a version of the Geometric Mechanism whose 
range is restricted to {0, . . . , n}, which will be easier to work 
with since it can be easily represented as a matrix. 

Definition 4 (Range- Restricted Geometric Mechanism). 
For a given privacy parameter a, when the true query re- 
sult is k € [0,n], the mechanism outputs Z(k) where Z(k) 
is a random variable with the following distribution for each 
integer z: 

Pr[Z(k) = z] = I i=| • al z - fc l if0<z<n 
I otherwise. 

This mechanism is equivalent to the geometric mechanism in 
the sense that we can derive this from the geometric mech- 
anism and derive the geometric mechanism from its range- 
restricted version. We shall refer to both as the Geometric 
Mechanism and denote the matrix by G„, a . (Table 2). 



For ease of notation, we shall denote by G' n ^ a the matrix 
obtained by multiplying the columns 1 and n of G n . a by 
(1 + a) and all other entries by jz^- Table 2 shows the 
matrices of G n , a and G' n}Cl . We are now ready to state the 
characterization. 

Theorem 2. Suppose M is any oblivious differentially pri- 
vate mechanism. Then M can be derived from the geomet- 
ric mechanism if and only if every three consecutive en- 
tries xi,X2,X3 in any column of M satisfy (X2 — axi) > 
a(x3 — ax2)- 

The key insight is to think of each column in M and in G n , a 
as a vector. Looking at the problem through this linear 
algebraic lens, we see that deriving M from G n , a amounts 
to proving that each column of M lies in the convex hull of 
the (vectors which form the) columns of G n , a - In Lemma 1, 
we show that G n , a is non-singular, hence each column of 
M can be represented as a linear combination of columns of 

G n ,a • 

Lemma 1. det(G„, a ) > 0. 

Proof. Since G' nt<x can be obtained by multiplying each entry 
in the first and last column of G„. Q by (1 + a) and entries in 
all other columns by f^|, detG^ = (l+a) 2 ( j^) n_2 det G, 
Hence, we only need to prove that det G' n ^ a > 0. We prove 
this by induction on n. For n = 2, we explicit calculation 
yields G^a = (1 — a 2 ). For the general case, perform the 
column transformation Gi <— C\ — aC2 on G' n a . Expanding 
on the first column gives us det G' niCt = (1 — a 2 ) det G' n -i, a . 
Hence, by induction, deiG' n a = (1 — a 2 )™ -1 . □ 

We need to show that each column of M is actually a convex 
combination of columns of G. We can write M = G„ i(J • 
T for some matrix T. Hence, T = G~* a ■ M. Note that 
G n , a and M are both generalized stochastic matrices. Since 
the set of all non-singular generalized stochastic matrices 
forms a group [19], Gn, a is a generalized stochastic matrix. 
And since generalized stochastic matrices are closed under 
multiplication, T is also a generalized stochastic matrix and 
is uniquely defined. All we need to prove is that all entries in 
T are non-negative. We shall use Cramer's Rule to calculate 
the entries of T and complete the proof. 

Given a n x n matrix G and a vector x — (xi, . . . , x n ) 1 , 
define G(i, x) as the matrix where the i th column of G has 
been replaced by x. 

Let tj be the j th column of T. ttj denotes the i,j entry 
in T. Observe that, G n ,a ■ tj = rrij. By Cramer's Rule, 
we get that tij = dct ^g('' m j) _ ^0 calculate this, we shall 
explicitly calculate the value of det G n , a (i, rrij). 
Lemma 2. Given G n , a and a vector x — (xi, . . . , x n ) 1 : 

1. det G„, (l, cc) > iff x\ > ax2 

2. detG„, a (n, x) > iff x„ > ctx n -\ 

3. detG n , a (i,x) > if and only if (X2 — axi) > a(x3 — 
ax2) : For 2 < i < n — 1 
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1 
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a 2 
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Table 2: The Range Restricted Geometric Mechanism 



Hence, when M satisfies the condition that for every three 
consecutive entries x\,xi,xz in any column [xi — axi) > 
a(x 3 — 0x2), then Sij > for all This proves that M 
can be derived from the geometric mechanism. 

To prove the converse, suppose that there is a column c and 
row i of M such that ((l+a 2 )mij — a(mi-ij+mi+ij)) < 0, 
then s»,c = det G(i, m c )/ det G < 0. This says that M can- 
not be derived from G. This completes the proof of Theo- 
rem 2. □ 



We now prove Lemma 2, using similar column transforma- 
tions as we used in Lemma 1 to calculate det G n , a (i, x) for 
an arbitrary vector x. 

Lemma 2. Given G n>a and a vector x = (xi, . . . , £„)'.■ 



1. det G„, a (l, x) > iff xi > 01x1 

2. det G n , a {n, x) > iff x n > ax n -i 

3. det G„, a (i, x) > if and only if (x 2 — axi) > a(x 3 — 
0x2) : For 2 < i < n — 1 



4. APPLICATIONS OF THE CHARACTER- 
IZATION 

We show two applications of the characterization result of 
Theorem 2. The first one gives us a way to simultaneously 
release data to consumers at different levels of privacy. As a 
second application we show how to obtain a optimal mech- 
anism for an information consumer without knowing its pa- 
rameters. 

4.1 Information-Consumers at Different Pri- 
vacy Levels 

Suppose we want to release the answer of the query to differ- 
ent information consumers. We represent the level of privacy 
of a consumer c by the privacy parameter a c . Given true 
result r, we will release r c to consumer c such that the mech- 
anism is a c -differentially private. We expect that consumers 
at different levels of privacy do not share query results with 
each other which is enforced via, say, non-disclosure agree- 
ments. Even when they do share data, we want our mech- 
anism to be collusion-resistant and not leak privacy- the 
colluding group should not get any more information about 
the database than the consumer with access to the least 
private result i.e., the one with the smallest a . 



Proof. We will prove the above properties for G' n ^ a . Since, 
G' nia is obtained from G„, a by multiplying columns with 
positive reals, the properties above will continue to hold for 
G n , a - We divide the proof into cases depending on the value 
of i : 



1. i = 1 : We repeatedly do the column transforma- 
tion C n C n — aCn-i to get that det G^ a (l, x) = 

(1 - a 2 )"" 2 ^ " = (1 - a 2 )"- 2 (:ri - ax 2 ). Hence, 

detG' n a (l,x) > <=> (xi > 0x2). 

2. i — n : We can do the same column transformations 
to get that det G' n>a {n, x) = (1 - a 2 )"' 2 



1 X n -i 

a x„ 



(1 — o? 2 ) n 2 (x n — ax n -i). Hence, det G'„ :0l (n, x) > 
<=> {x„ > ax n -i). 

3. 2 < i < n — 1 : Similarly, for the general case we get 



that detG'„ ia (i,x) = (1 - a 2 )' 



1 Xi-i a' 



a Xi+i 



(1 - a 2 ) n 2 ((1 + ct 2 )x l - a(xi-i + x i+ i)). Hence, 
det G' nta (i, x) > <=> (x 2 — axi) > a(x3 — 0x2). 



□ 



We now describe a mechanism that achieves this. The next 
lemma gives us a way to "add" more privacy to an existing 
geometric mechanism. 

Lemma 3. For two privacy parameters a < f3, the geo- 
metric mechanism G n ,p can be derived from the mechanism 



G n 



there exists a stochastic matrix T a g such that 



G?i^Q — G n ,a ' T a ^ 



Proof. Theorem 2 states that G„ : g can be derived from G n , a 
if and only if for every three consecutive entries xi,x 2 , £3 in 
any column of G n>l g, (X2 — axi) > a(xz — 0x2). We check 
this condition for each of the three forms that consecutive 
entries in each row of Ge,n can have: 



1. (/3 i ,/3 i+1 ,/3'+ 2 ) : (1 + q 2 )/? i+1 -q(/3 1 + /? 1 + 2 ) = + 
a 2 /3-a- a/3 2 ) = fi\p - a)(l - a/3) > 0. 

2. 03, l,/3) : (1 + a 2 )l - a(/3 + /?) = 1 + a 2 - 2a/3 > 
(l-a) 2 > 0. 

3. (/3 ,+2 ,/3 I+1 ,/3 l ) : (l + a 2 )/3 l+1 -a(/3 ! +/3 l+2 ) = /3 l (/3 + 
a 2 /3 - a - a/3 2 ) = /3 l (/3 - a)(l - a/3) > 0. 



This shows that T Qi( 3 = G„ >ct • G n ,B is a stochastic matrix. 

□ 
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Algorithm 1: Releasing Query Result to Consumers at 
Multiple Levels of Trust. 

ht Input: True Query Result r. k privacy levels given by 

parameters cti < a.2 < • • • < ct k . 
Output: Query Results ri,r2, . . . , r k to be released. 
Define T\ = G ai ,n- 
for 1 < i < k do 

Compute post-processing matrix T i+ i such that 

end 

By Lemma 3, each T; is a stochastic matrix. Hence, we can 

think of Ti as a mechanism - Given any input k we sample 

from the probability distribution given by the k th row of T< 

which we represent by Ti(k) 

Let ro = r. 

for 1 < i < k do 

ri — Ti(n-i) is obtained by treating n-i as the true 
query output and applying mechanism Ti to it. 

end 

Release the query results n, V2, ■ ■ ■ , r k to consumers at 
privacy levels ai, . . . ,a k . 



The release mechanism is given in Algorithm 1. We con- 
clude the section by proving that Algorithm 1 is collusion- 
resistant. 

Lemma 4. Any subset C = {ci < • ■ • < c t } C {1, . . . , k} 
of colluding information consumers who have access to query 
results R(C) = {r ci . . . r Cfc }, released at privacy levels ct ci , ■ ■ ■ , Q c 
respectively, can only reconstruct as much information about 
the database d by combining their results as ci can working 
alone. 

Proof. The matrix G n , ai and post-processing matrices T ac . lQ 
can be calculated by anyone. Hence, given the random coin 
tosses made by the algorithm, Lemma 3 shows that r Cj can 
be obtained from r Ci for Cj > c;. Given r ci , having access to 
R(C) can at most reveal information about these coin tosses 
that Algorithm 1 made. Since, these coin tosses do not de- 
pend on the database, any information about the database 
that is reconstructed from R(C) can also be reconstructed 
by consumer ci (who has access to result r ci ) alone. □ 

4.2 Universal Utility Maximizing Mechanisms 

We now prove that if we deploy the geometric mechanism 
(Definition 4), then the interaction of every information con- 
sumer will yield a mechanism that is optimal for that con- 
sumer. Since, the geometric mechanism is not dependent 
on any information consumer's loss-function or side infor- 
mation, it is simultaneously optimal for all of them. 

Our result proves that all optimal mechanisms can be de- 
rived from the geometric mechanism. However, there do 
exist differentially private mechanisms (which are not opti- 
mal for any information consumer) that cannot be derived 
from the geometric mechanism. We give an example of such 
a mechanism in Appendix B. 

The first part of the proof shows that every two adjacent 
rows of every optimal mechanism must satisfy certain con- 
dition; if it does not, we can perturb the mechanism in a 



way to yield a differentially private mechanism with strictly 
better utility. The second part of the proof leverages this 
lemma and the characterization from Theorem 2 to complete 
the proof of Theorem 1. 

Lemma 5. For every monotone loss-function L(l, S) = 
maxjgs 2Z rgiv i(i, r) • Xi <r , there exists an optimal mecha- 
nism x such that for every two adjacent rows i,i + 1 of this 
mechanism, there exist column indices c\ and C2 such that: 

1. Vj G I...C1 : axij = Xi+ij 

2. V? G c 2 ...n : Xij = axi+ij 

3. Either C2 = ci + 1 or C2 = ci + 2. 

Proof. We define the function L' : M — > R given by L'(x) = 
Sigjv X^rgjv Xi > r ' N ~ r \- Consider the total order >- on E 2 
given by (a, 6) >- (c,d) <=> {(a > c) or (a = c and b > 
d)}. Let x be an optimal mechanism for the loss- function 
(L, L') according to the order defined above. The idea here 
is that there are multiple mechanisms that optimize L and 
using L' we isolate the ones with the property that we want. 
We prove by contradiction that x satisfies the constraints 
given above. 

Assume otherwise. Then there exist rows i, i+1 and columns 
j,k; k > j such that ax it j < x i+ ij and ax i+ i >k < x iik - 
We shall construct a differentially private mechanism y for 
which (Ly,L' y ) is strictly smaller than (L X ,L' X ) which is a 
k , contradiction since we assumed that x minimized (L,L'). 

We divide the proof into two cases : i < (j + k)/2 and 
i > (j + k)/2. Consider the case i < (j ' + k)/2 first. For 
1 G {1 ... i} set Xi'j+Sxi'^ and yi> ik {l — 8)xi> t k- 

For all other values set yi, m = xi tTn . We first show that y is 
1 a differentially private mechanism. Let the set of changed 
elements C = {yi >m '■ m G {j, k} and I < i}. The set of 
unchanged elements U is all the remaining yi.m- All privacy 
constraints involving elements only from U are satisfied since 
they were satisfied in M. The privacy constraints involving 
only elements in C continue to hold since they are the same 
linear combinations of corresponding elements from M. We 
only need to check that the privacy constraints are satisfied 
when one element is from C and another from U. But this 
only happens for yi,j,yi+i,j and yi, k ,yi+i,k- By assumption, 
axij < Xi+i t j and axi+i ;k < x i)k . We can choose a small 
enough 8 such that aytj = a(xij + 8x iyk ) < x i+ ij — 
and m/;,fc = q(1 — S)xi,k < = S/»+i,fc- Also, for m G 

{j, k}, y itm > x i;rn > ax i+ i, m = Vi+i, m ■ This proves that 
y satisfies differential privacy. 

Now, we shall prove that y has strictly smaller loss than x. 
For any row r G {1, . . . ,i}, the change in loss due to row r 
is 

^2 l(r, i)x r ,i - l ( r > i)Vr,i 

i£N i€N 

= {l( r J) x r,j +l{r,k)x r ,k) - (l(j-,j)y r ,j + l{r, k)y r ,k) 
= (l(r,j)x r ,j +l{r,k)x r , k ) 

- (l(r,j)(x rJ + 8x r>k ) + l(r, k)(x rzk - 8x r , k ) 
= 8x r:k (l(r, k) - l(r,j)) 
> since l(i,j) is monotonic in \i — j\. 
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The total loss L = max rS ,s^\ 

L x > L y . AISO Xi.Er 1 ',' 



eJV 
i — r 



l(i,r) ■ Xi, r and since so 

This means that (L x , L' x ) >- (L y ,L' y ). But a; was an optimal 
mechanism with respect to >-. This gives us a contradiction. 



The proof for the case i > (j + k)/2 is similar. For i > 



set yi' ik 



+ Sxi'j and t/j/j «— (1 — S)xi'j. The 



same arguments as above now hold for this definition of y 
as well. □ 



We are now ready to prove Theorem 1. We state it again 
for convenience. 

Theorem 1. Consider a database d, count query q, k con- 
sumers and privacy levels on < ... < ah- There exists a 
mechanism M that constructs k results n . . .ru, and releases 
result n to the ith information consumer, such that: 



1. ( Collusion- Resistance) Mechanism M is a t i -differentially 
private for any set I of colluding information con- 
sumers who combine their results. Here, C C {1 ... k} 
and i' = min-{j : j £ C}. 

2. (Simultaneous Utility Maximization) Suppose that the 
ith consumer is rational and interacts optimally with 
the mechanism (as described in Section 2.4-3), then 
its utility is equal to that of the differentially private 
mechanism tailored specifically for it (the mechanism 
from Section 2.5). 



c — ~Y2 l>k x it i. Rewrite Equation (2) to get: < a?i+ij — 
ctx i+ 2,j < a(xij — ax i+ ij) => Xij > axi+ij Thus, by 
Lemma 5, k > j. We now claim that: 



(l + a 2 )b' -a(b + b") < 



(3) 



This is true from Equation (2) if fe — j. Otherwise rewrite 
Equation (2) to get < Xi+i,j—otXi,j < a(xi+2,j—axi+i ; j) = 
x i+2,j > otXi+i,j. Thus, by Lemma 5, it must be that 
a ■ b" = b, Further, by privacy b > ah' and so, b > a 2 b' . 
This proves the claim. 

Because M is a generalized stochastic matrix, Y^i x i,i = 
2; x i+i,i = 1- Thus, a + b + c — 1 and a ■ a + b' + c/a = 1. 
Using these equations, we have: 

1 — b — a + b'ct a — a 2 + ba 2 — b'a 

a = and c = (4 

1 — a 1 1 — a. 1 

We now prove that M is not feasible. 

^ Xi + 2,i > a 2 ■ a + b" + c/a 2 
i 

_ a 3 - ba 3 - a 4 + b'a A + b"a - b"a 3 + l-a-b' + ba 
a(l - a 2 ) 



1-a + a 2 (b + b")a-b'(l + a 2 ) 



> 1 



Proof. Algorithm 1 is used to deploy geometric mechanism 
at different levels of privacy. Lemma 3 shows that it is al- 
ways possible to deploy geometric mechanism this way. This 
proves that the deployed mechanisms are differentially pri- 
vate. Lemma 4 proves that the release is a c ., -differentially 
private even for any set C of colluding consumers, where 
i' = minjj : j G C}. This completes the proof of part 1. 

To prove part 2, we concentrate on a single trust level with 
privacy parameter a. We prove the result by contradic- 
tion. Assume there is an information consumer c with loss- 
function I and side-information S, whose interaction with 
G n ,a does not optimize its loss. Let M be an optimal dif- 
ferentially private mechanism for c that satisfies Lemma 5. 
Since, c cannot optimize its loss by interacting with G n , a , 
M cannot be derived from the geometric mechanism. Wc 
prove that this implies that M is infeasible which is a con- 
tradiction. 

We know from Theorem 2 that there exists a column j 
of M and rows i, i + l,i + 2, such that the three entries 
Xi,j,Xi+i,j,Xi+2,j satisfy 



(f + a 2 )x i+1J - a(x i} j + x i+ 2,j) < 0. (2) 

Recall the pattern of every pair of adjacent rows of M from 
Lemma 5. Let k be the unique column that satisfies axi,k < 
Xi + i : k and ax i+ i yk < x i>k , or if there is no such column, 
let it be the last column such that Let 

a = J2i<k Xi > l > b = Xi ' k > ^ = x i+i,k, b" = x i+2 , k and 



The first step is from Equation (2) and Lemma 5, the second 
is by Equation (4), the third is by rearranging and the fourth 
holds because the first summand is always at least 1 and the 
second is strictly positive by Equation (3). □ 

5. CONCLUSION 

We give a minimax model of utility for information con- 
sumers that is prescribed by decision theory. We show that 
for any particular count query, the geometric mechanism 
is simultaneously optimal for all consumers, assuming that 
consumers interact rationally with the output of the mech- 
anism. This is particularly useful in publishing aggregate 
statistics, like the number of flu infections in a given region, 
to a wide unknown audience, say on the Internet. 

An open question is to investigate whether similar guar- 
antees are possible for multiple queries and other types of 
queries. 
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APPENDIX 

A. THERE ALWAYS EXISTS AN OPTIMAL 
MECHANISM THAT IS OBLIVIOUS 

In Section 2 we restricted attention to oblivious mechanisms. 
While natural mechanisms (such as the Laplace mechanism 
from [5]) are usually oblivious, we now justify this restriction 
from first principles. Specifically, we show that for every in- 
formation consumer with a loss-function over databases and 
side information over query results, there exists a oblivious 
loss-function and side-information, such that the optimal 
utility with the oblivious loss-function is no more than the 
optimal utility with the non-oblivious loss-function. 

Consider a non-oblivious mechanism x. For the minimax 
information consumer with loss-function I over databases 
and side information S C {0, 1, ... , n}, the utility of this 
mechanism is given by 



max 



x d,r ■ l(f(d),r) 



(5) 



rgJV 



The following lemma proves that obliviousness is without 
loss of generality i.e. there always exists an oblivious mech- 
anism whose loss is lower than or equal to the loss of the 
best non-oblivious mechanism. 

Lemma 6. Fix a database size n > 1 and privacy level a. 
For every minimax information consumer with loss-function 
I and side information S C {0,1,..., n}, there is an a- 
differentially private mechanism that minimizes the objective 
function (5) and is also oblivious. 
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Proof. We shall now construct a differentially privacy mech- 
anism x' that is oblivious and whose loss is not greater than 
the loss of x. This will prove our assertion. 

We construct a partition E of all the databases, according to 
the query output. All databases that have the same query 
output belong to the same subset of the partition. For a 
database d, let E(d) = {d' : q(d) = q(d')}. For r G N and 
d G D n , define x' E ^ r = £tvg d , eE ( d - ) x d ^ r . It is clear that x' 
is an oblivious mechanism. 

First we show that x' is a-differentially private. Fix two 
databases di,d2 G D n such that d\ and di differ in exactly 
one row; We need to show that ax' dir < x' d2r . Assume 
f(di) ^= /(efe), otherwise the proof is trivial. 

For any database of E(di), we can generate all its neigh- 
bors (databases that differ in exactly one row) in E(d,2) by 
enumerating all the ways in which we can change the query 
result by exactly 1. For instance when f(di) = /(<fe) + 1, 
pick one of the n — f(di) rows that satisfy the predicate in di 
and change its value to one of those that violates the pred- 
icate. This process is identical for all databases of E(di), 
and so for all d G E{d\), the number of neighbors of d that 
belong to the set E{di) is the same (does not vary with d). 
Similarly, for all d G £(<&), the number of neighbors of d 
that belong to the set E(di) is the same. 

Consider the following set of inequalities that hold because 
x is a-differentially private: d G E(di), d! G E(d,2), where 
di and di are neighbors, axdr < x d / r . By the argument 
in the above paragraph, all the databases in E{d\) appears 
equally frequently in the left-hand-side of the above inequal- 
ity and all the databases in E{d2) equally frequently in the 
right-hand-side. Summing the inequalities and recalling the 
definition of x' completes the proof of privacy. 

Now we show that x' does not incur more loss than x. The 
loss for x' is given by max d65 ci3« E reJV x E(d),r " l if( d ), r )- 
Suppose the worst loss for x' occurs for the partition E(di). 

L ( X ') = X B(dl,r) • Kf( d l), r ) 

reN 

= av SdeE(d 1 )( x d,r) ■ l(f(d),r) 

reN 

deE{dl KeN 
< max x dr ■ l(f(d),r) = L(x). 

deSCD" t-^i 
reN 



private. 

" 1/9 2/9 4/9 2/9" 

2/9 1/9 2/9 4/9 

4/9 2/9 1/9 2/9 

13/18 1/9 1/18 1/9 

We claim that M cannot be derived from the geometric 
mechanism. We can explicitly calculate G~\ ■ M to see 

,3 ' 2 

that M is not derivable from the geometric. Instead we 
shall use the characterization from Theorem 2. If we look at 
elements M (0, 1), M(l, 1), M(2, 1) , then (1 + a 2 )M(l, 1) - 
a(M(0, 1) + M(2, 1)) = 1.25 x | - \ x (§ + §) = This 
proves that M cannot be derived from G 3 i. 



This completes the proof. 

□ 



B. A DIFFERENTIALLY PRIVATE MECH- 
ANISM THAT IS NOT DERIVABLE FROM 
THE GEOMETRIC MECHANISM 

Consider the mechanism M given by the following matrix. 
M(i,j) gives the probability of returning j when the true 
query result is i. We can verify that M is \ -differentially 
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